Electronic apparatus having a sensing unit to input a user command and a method thereof

ABSTRACT

A mobile device includes a housing, a network interface to communicate with an external network, a user interface comprising a panel with a display panel to display a menu image and a touch panel to receive a touch input, the panel including a first portion and a second portion with a transparent condition and being within the first portion, a sensing unit including two front cameras disposed in the second portion of the panel, and a control unit to control the sensing unit in a photographing mode and a sensing mode to perform a function of the mobile device.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a Continuation application, under 35 U.S.C. § 120,of U.S. Non-Provisional application Ser. No. 17/404,381, filed Aug. 17,2021, which is a Continuation application Ser. No. 17/071,042, filed onOct. 15, 2020, now U.S. Pat. No. 11,182,021, which is a continuationapplication Ser. No. 16/528,695, filed on Aug. 1, 2019, now U.S. Pat.No. 10,845,922, which is a Continuation application of a Non-Provisionalapplication Ser. No. 15/875,367, filed on Jan. 19, 2018, now U.S. Pat.No. 10,423,284, which is a Continuation application of a Non-Provisionalapplication Ser. No. 15/440,033, filed on Feb. 23, 2017, now U.S. Pat.No. 10,013,115, which is a Continuation application of a Non-Provisionalapplication Ser. No. 15/246,302, filed on Aug. 24, 2016, now U.S. Pat.No. 9,864,511, which is a Continuation application of a Non-Provisionalapplication Ser. No. 14/583,537, filed on Dec. 26, 2014, now U.S. Pat.No. 9,454,235, in the United States Patent and Trademark Office, thecontents of which are incorporated herein in their entirety byreference.

BACKGROUND OF THE INVENTIVE CONCEPT 1. Field of the Inventive Concept

The present inventive concept relates to an electronic apparatus andmethod of inputting a user command through a sensing unit thereof.

2. Description of the Related Art

A conventional electronic mobile device, for example, a cellular phone,includes a camera and a display unit having a display area to display animage photographed by the camera. However, a center line of the cameradoes not correspond to a center line of the display area. Therefore,when an object is disposed over a position of the center line of thedisplay area, an image of the object is displayed on a positiondifferent from the center line of the display area of the display unit.

When a user intends to place an object on a position to be disposed overthe display area to correspond to a position of the display area in avertical direction relationship between the position of the object andthe position of the display area, an image of the object displayed onthe display area is deviated from the intended position.

When a user intends to move an object between positions to be disposedover the display area to correspond to positions of the display area ina vertical direction relationship between the positions of the objectand the positions of the display area, an image of the object displayedon the display area is deviated from the intended positions.

Since a position of an object is sensed to be different from a positionof a display area, it is difficult to sense a position of an objectsensed by a camera to correspond to a position of a point of a displayarea.

SUMMARY OF THE INVENTIVE CONCEPT

The present inventive concept provides a mobile device including a panelhaving a first portion and a second portion within the first portion,and a sensing unit having two front cameras installed in the secondportion of the panel to sense a user over the panel through the secondportion in a photographing mode and a sensing mode.

The present inventive concept provides a method of a mobile devicehaving a panel having a first portion and a second portion, and asensing unit having two front cameras in the second portion, the methodincluding sensing a user over the panel through the second portion in aphotographing mode and a sensing mode.

The present inventive concept provides a computer readable medium tocontain computer readable codes as a program to execute a method of amobile device having a panel having a first portion and a secondportion, and a sensing unit having two front cameras in the secondportion, the method including sensing a user over the panel through thesecond portion in a photographing mode and a sensing mode.

Additional features and utilities of the present inventive concept willbe set forth in part in the description, which follows and, in part,will be obvious from the description, or may be learned by practice ofthe present inventive concept.

The foregoing and/or other features and utilities of the presentinventive concept may be achieved by providing a mobile device includinga housing, a network interface to communicate with an external network,a user interface comprising a panel with a display panel to display amenu image and a touch panel to receive a touch input, the panelcomprising a first portion and a second portion with a transparentcondition and being within the first portion, a sensing unit comprisingfirst and second front sensing elements which are disposed in the secondportion of the panel to sense an object distanced from the panel throughthe second portion, and a control unit to control the sensing unit in aphotographing mode and a sensing mode to perform a function of themobile device.

The foregoing and/or other features and utilities of the presentinventive concept may also be achieved by providing a mobile deviceincluding a housing, a network interface to communicate with an externalnetwork, a user interface including a panel with a display panel todisplay a menu image and a touch panel to receive a touch input, thepanel having a first portion and a second portion in the first portion,the second portion having an interference-prevention condition, asensing unit including first and second front cameras disposed in thesecond portion of the panel to sense an object over the panel throughthe second portion, and a control unit to control the sensing unit in aphotographing mode and a sensing mode to perform a function of themobile device.

The object may be distanced from the panel.

The first portion and the second portion may be parallel to each otherwithin the panel.

The second portion may be smaller than the first portion with respect toa display area of the panel.

The second portion may be surrounded by the first portion.

The second portion may include opposite sides with the first portion.

The first portion may include light elements, and the second portion maynot include the light elements.

The first portion may include a liquid crystal device.

The first portion may include a light emitting device.

The first portion may include a first number of light elements, and thesecond portion may include a second number of light elements.

The interference-prevention condition may be a transparent condition.

The interference-prevention condition may prevent the first and secondfront cameras from being interfered during sensing the object in thephotographing mode and the sensing mode.

The first front camera may photograph the object through the secondportion in the photographing mode.

The control unit may control the display panel to display thephotographed object.

The second front camera may sense the object in the sensing mode.

The control unit may control the display panel to display a symbolrepresenting a status of the object.

The first front camera and the second front camera sense the object as auser input in the sensing mode.

The sensing unit may be activated to photograph and extract an objectimage from the photographed image as a user input to perform thefunction of the mobile device in the sensing mode.

The sensing unit may selectively perform the sensing mode and thephotographing mode.

The control unit may control the first and second front cameras toselectively perform the sensing mode and the photographing mode.

The object may include a plurality of sub-objects, and the control unitmay perform a second function of the mobile device according to onesensed sub-object with a priority.

The object may include a user having a plurality of portions, one ofwhich has a higher priority, and the control unit may perform a secondfunction of the mobile device according to one sensed sub-object withthe higher priority.

The object may include a user having a plurality of portions, one ofwhich has a priority, and the control unit may perform a second functionof the mobile device according to the sensed user and the sensed oneportion with the priority.

The first front sensing camera may include a first lens, the secondfront may include a second lens, and the first lens and the second lensmay be disposed in the second portion.

The first lens and the second lens may have a sensing direction parallelto a line perpendicular to a display surface of the panel.

At least one of the first lens and the second lens mat have a sensingdirection which is not parallel to a line perpendicular to a displaysurface of the panel.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other features and utilities of the present generalinventive concept will become apparent and more readily appreciated fromthe following description of the exemplary embodiments, taken inconjunction with the accompanying drawings of which:

FIG. 1 is a diagram illustrating an electronic apparatus according to anembodiment of the present inventive concept;

FIG. 2 is a block diagram illustrating an electronic apparatus accordingto an embodiment of the present inventive concept

FIGS. 3 and 4 are views illustrating a sensing unit and a display screenin an electronic apparatus according to an embodiment of the presentgeneral inventive concept;

FIGS. 5 and 6 are views illustrating an object sensed by a sensing unitof an electronic apparatus according to an embodiment of the presentinventive concept;

FIGS. 7 and 8 are views illustrating a point of a virtual area tocorrespond to a display point of a display area in an electronicapparatus according to an embodiment of the present inventive concept;

FIGS. 9 and 10 are views illustrating a status of an objectcorresponding to a virtual point with respect to a virtual area and adisplay area in an electronic apparatus according to an embodiment ofthe present inventive concept;

FIGS. 11 and 12 are views illustrating objects sensed by a sensing unitin an electronic apparatus according to an embodiment of the presentinventive concept;

FIG. 13 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept;

FIGS. 14A, 14B, 15A, and 15B are views illustrating a virtual area of asensing unit and a display area of a display unit according to anembodiment of the present inventive concept;

FIG. 16 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept;

FIG. 17 is a view illustrating a virtual area of a sensing unit and adisplay area of a display unit according to an embodiment of the presentinventive concept;

FIGS. 18A and 18B are views illustrating a virtual point displayed on adisplay area of a display unit in an electronic apparatus according toan embodiment of the present inventive concept;

FIGS. 19 and 20 are flowcharts illustrating a method of inputting a usercommand using a sensing unit in an electronic apparatus according to anembodiment of the present inventive concept;

FIG. 21 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept; and

FIGS. 22 and 23 are views illustrating an electronic apparatus having asensing unit installed thereon according to an embodiment of the presentgeneral inventive concept.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Reference will now be made in detail to exemplary embodiments of thepresent general inventive concept, examples of which are illustrated inthe accompanying drawings, wherein like reference numerals refer to thelike elements throughout. The exemplary embodiments are described belowin order to explain the present general inventive concept whilereferring to the figures.

FIG. 1 is a diagram illustrating an electronic apparatus 100 accordingto an embodiment of the present inventive concept. The electronicapparatus 100 may be a mobile device having one or more camera units tophotograph an object disposed in a front side and/or a back side of themobile device. The electronic apparatus 100 includes a housing 101, adisplay unit 102 disposed on the housing 101 to display an image on adisplay area DA having a display center line DCL, and a sensing unit 103having a sensing element to sense an object disposed around theelectronic apparatus 100.

The sensing unit 103 may include the one or more camera units having asensing center line SCL to generate an image corresponding to thephotographed object with respect to the display area DA. The sensingcenter line SCL and the display center line DCL are spaced apart fromeach other. Although the sensing center line SCL is illustrated in aleft-right direction of FIG. 1 , the sensing center line SCL may bedisposed at a line perpendicular to a surface of the housing 101, forexample, the display area DA. Although the display center line DCL isillustrated in a left-right direction of FIG. 1 , the display centerline DCL may be disposed at a line perpendicular to a surface of thehousing 101, for example, the display area DA.

As an example, when the sensing unit is a camera to photograph anobject, and the display area DA displays an image corresponding to thephotographed object, a center of the displayed image of the photographedobject may be not identical to a center of the photographed object dueto a distance between the sensing center line SAC and the display centerline DCL. When an object approaches toward a portion of the display areaDA in a direction perpendicular to a surface of the display area DA, theimage of the photographed object is not displayed to move toward theportion of the display area DA in a direction perpendicular to a surfaceof the display area DA. As another example, when an object moves towardor disposed at a center of the display area DA, the image of thephotographed object is not displayed at the center of the display areaDA but is displayed at another portion spaced apart from the center ofthe display area DA.

When a user is disposed in front of the electronic apparatus to face thedisplay area DA and a user finger or tool is disposed over or movestoward a point of the display area DA, a photographed image of the userfinger or tool is displayed on the display area DA differently as if theuser finger or tool is disposed over or moves toward another point ofthe display area DA. The photographed position (location) or movement ofthe user finger or tool may be referred to a virtual position (location)or movement of the object, and the position (location) or movement ofthe object with respect to the display area DA may be referred to anactual or physical position (location) or movement of the object. Thevirtual position or movement of the object in the photographed or sensedimage with respect to the display area DA may be different from theactual position or movement with respect to the display area DA.Therefore, the electronic apparatus may not determine or recognize thestatus of the object as a user input according to the virtual positionor movement in the photographed or sensed image but may generate anerror in determining or recognizing the virtual position of movement ofthe object as a user input with respect to the display area DA.Accordingly, the virtual position (location) or movement of thephotographed image of the object may be modified or corrected to theactual or physical position (location) or movement such that the statusof the object can be determined as a user input with respect to a pointof the display area DA. This modification or correction allows theelectronic apparatus to determine or recognize a user input from thephysical location or movement of the object with respect to the displayarea DA.

FIG. 2 is a block diagram illustrating an electronic apparatus 200according to an embodiment of the present inventive concept. Theelectronic apparatus 200 may be the electronic apparatus 100.

The electronic apparatus 200 includes a controller 210, a memory unit220, an interface 230, a sensing unit 203, and an input/output unit 204.The controller 210, the memory 220, the interface 230, the sensing unit203 and the input/output unit 204 may be included in a single housing.However, it is possible that at least one of the memory 220, theinterface 230, the sensing unit 203 and the input/output unit 204 may beconnected to the housing. The sensing unit 203 may correspond to thesensing unit 103 of FIG. 1 , and the input/output unit 204 maycorrespond to the display unit 102 of FIG. 1 . The memory unit 220 maystore data usable to perform functions of the electronic apparatus 200.The memory unit 220 may be a semiconductor memory device, a read-onlymemory (ROM), a random-access memory (RAM), a flash memory, asemiconductor chip package, CD-ROMs, magnetic tapes, floppy disks, andoptical data storage devices. The input/output unit 204 may be a displaydevice to display an image or a touch panel to receive a user input. Theinput/output unit 204 may include an audio element to generate an audiosignal alone or together with an image displayed on a display unitaccording to a user input or data processed in the controller 210.

The sensing unit 203 may include one or more cameras to photograph anobject and/or a sensing element to sense an object disposed over adisplay area of the display device or touch panel. The photographed orsensed object may be displayed as an image on the display area DA.However, it is possible that the photographed or sensed object may notbe displayed as an image but may be usable as a user input (command ordata) to correspond to a portion (icon or menu) of the display area DA.It is also possible that the photographed or sensed object may not bedisplayed as an image of the photographed object but may be displayed asa symbol representing the photographed or sensed object with respect tothe display area DA. That is, when the sensing unit 203 is in a cameramode, the photographed object is displayed as an image on the displayarea DA. When the sensing unit 203 is in a sensing mode, the sensedobject is usable as a user input (or user data input) to correspond to aportion (icon or menu selection or a user command) of the display areaDA. It is also possible that in a combination mode, the photographedobject is displayed on the display area DA, the icon or menu isdisplayed on the display area DA, and the photographed (or sensed)object is used as a user input to control or perform a function of theicon (or menu) of the display area or to control the displayed image orto control the sensing unit 203. The camera mode, the sensing mode, andthe combination mode can be selected in a menu screen of the displayarea DA or can be selected according to a selection of a buttoncorresponding to the respective mode according to a user input.

The input/output unit 204 may be a display unit to display an image,and/or an input unit to input a user input (command or data). Theinput/output unit 204 may have a display area DA of a screen (panel) todisplay an image photographed by a camera, to display a menu icon, toreceive a user touch as a user input through the displayed menu icon,and to display an image sensed by the sensing element. The panel may bea touch panel. The touch panel is well known, detail descriptionsthereof are omitted. The input/output unit 204 may separately orindependently generate or receive the photographed image, the displayedmenu icon, the received user input, and the sensed image incorresponding modes according to a user design or preference. It ispossible that the input/output unit 204 may simultaneously generate orreceive the photographed image, the displayed menu icon, the receiveduser input, and the sensed image in corresponding modes according to auser design or preference. It is also possible that the input/outputunit 204 may generate a combination of the photographed image, thedisplayed menu icon, the received user input, and the sensed imageaccording to a user design or preference.

The controller 210 may generate a menu option to be displayed on thedisplay area DA. The menu option may include a first menu option toselect a camera mode in which a camera photographs an object andgenerates the photographed image to be displayed on the display area DA,a sensing mode in which a sensing unit senses an object disposed overthe display area DA and generates the sensed image corresponding to thesensed object to be displayed on the display area DA, and a combinationmode in which a camera is usable as a camera and a sensing element toperform both functions of the camera and the sensing element. Here, thephotographed image is an image photographed by a camera and the sensedimage is a symbol of the sensed object. Therefore, when one or moreobjects are photographed, the photographed objects are displayed on thedisplay area DA, and at least one of the photographed objects may besensed as the sensed image to be displayed as a symbol representing atleast a portion of the one object. The photographed image and the sensedimage may overlap on the display area DA. And also the menu icon and thesensed image may overlap on the display area DA. The sensed image may beuseable as a user input corresponding to a menu icon displayed on thedisplay area DA of the screen so that the controller 210 can determinethat a menu icon is selected according to a location of the sensed imageand can perform a function of the menu icon corresponding to the sensedimage.

The controller 210 may receive the photographed (or sensed) object fromthe sensing unit 203 and generate an image corresponding to thephotographed object to be displayed on the display area. Here, it ispossible that the controller 210 controls or modifies the image todisplay a portion of the photographed object which is disposed above thedisplay area DA in a direction, for example, in a directionperpendicular to the display area DA such that a center line of thedisplayed portion of the photographed object is identical to the centerDCL of the display area DA or such that an area of the display portionof the photographed object is identical to the display area DA. Thesensing unit 203 may perform at least a portion of the above-describedfunction of the controller 210. According to status of the photographedor sensed object, the controller may receive a user input or command toperform a function including, but not limited to, a drag operation, aselecting operation, a data input operation, a displayed imagemodification operation, an communication method selecting operation, adata transmission or reception method selecting method, an externaldevice selecting operation, an access point selecting operation, and soon. The status of the object may be a stationary or a motion between twopositions in a virtual area VA of the sensing unit 203 with respect tothe display area DA of the input/output unit 204.

As stated above, due to different center lines of the display area DAand the sensing unit 203, a location or movement of the photographedobject with respect to the display area DA is displayed at a differentlocation or movement within the display area DA. When an object isdisposed at a position over a center of the display area DA in adirection perpendicular to the display area DA, the photographed objectis displayed at a different position from the center of the display areaDA. However, the photographed object disposed at a position over acenter of the display area DA in the perpendicular direction may bedisplayed at the center of the display area DA by analyzing andmodifying a photographed object image. The controller 210 may control,adjust, or modify a location or movement of an object photographed withrespect to the sensing unit 203 to correspond to a location and amovement with respect to display area DA. In this case, an objectdisposed at another portion which is not in an area corresponding to thedisplay area DA in the perpendicular direction may not be displayed inthe display area. An object disposed in an area corresponding to thedisplay area DA in the perpendicular direction may be displayed in thedisplay area. The sensing unit 203 may perform at least a portion of theabove-described function of the controller 210.

The controller 210 may include a look-up table usable to analyze a firststate of the object in the photographed or sensed object in the virtualarea VA and to convert (correct or modify) the first state of the objectinto a second state of the object with respect to the display screen DAas the status of the object. The look-up table may include informationon coordinates of the virtual area VA and the display screen andinformation on a virtual location (or movement) of the object in thephotographed or sensed image in the virtual area VA and an actual orphysical location (or movement) of the object with respect to thedisplay area DA. When the object is disposed or moved in the virtualarea VA, the sensing unit 203 photographs the object to generate thephotographed or sensed image of the object with respect to thecoordinate of the virtual area VA. And then a location or movement ofthe photographed or sensed image of the object is analyzed and modifiedto a different location or movement with respect to the coordinate ofthe display screen. Since the coordinates of the virtual area VA and thedisplay area DA are same, the object is represented in the coordinate ofthe display area DA at a different location or movement from thelocation or movement in the virtual area VA. Although the coordinates ofthe virtual area VA and the display area DA are same, the virtuallocation or movement of the object photographed or sensed with respectto the virtual area VA may be different from the actual or physicallocation or movement of the object with respect to the display area DAaccording to a distance between the sensing center line SCL of thesensing unit 203 and the display center line SCL of the input/outputunit 204 or according to a distance from the sensing center line SCL tothe location of the object in the virtual area VA.

The interface 230 communicates with an external device 260 using a wiredor wireless communication method to transmit data to the external device260 and to receive data from the external device 260 according to a userinput. The controller 210 controls the above described units andelements of the electronic apparatus 200. The controller 210 may includeone or more semiconductor processors to perform functions of theelectronic apparatus 200.

FIG. 3 is a view illustrating a sensing area SA and a display area DA inan electronic apparatus according to an embodiment of the presentgeneral inventive concept. A camera (or sensing element) C1 as thesensing unit 203 of FIG. 2 photographs an object in the sensing area SAhaving a photographing angle “a” and a sensing center line SCL. Avirtual area VA is disposed in the sensing area SA to correspond to thedisplay area DA in a direction perpendicular to the display area DA. Thesensing center line SCL of the camera C1 may be parallel to a displaycenter line DCL of the display area DA. It is possible that sensingcenter line SCL of the camera C1 may not be parallel to a display centerline DCL of the display area DA to control or adjust a distance d1 whichis a distance between the display area DA and a boundary of a virtualplane VP or a boundary SAb of the sensing area SA. The distance D1 maybe adjustable when the camera C1 is adjustably installed in the housing101 of FIG. 1 to change the sensing center line SCL.

FIG. 4 is a view illustrating a sensing unit and a display screen in anelectronic apparatus according to an embodiment of the present generalinventive concept. The sensing unit 203 may have the camera C1 of FIG. 3to have a first sensing center line SCL1 and a first sensing area SA1,and a camera C2 to have a second sensing center line C2 and a secondsensing area SA2. The virtual plane VP may be disposed in an overlaparea of the first sensing area SA1 and the second sensing area SA2. Thevirtual plane VP may be disposed between the first sensing center lineSCL1 and the second sensing center line SCL2. The virtual area VA isdisposed over the display area DA in a perpendicular direction withinthe overlap area. the virtual plane VP may be spaced apart from thedisplay area DA by a distance d2. The distance d2 may also be adjustableaccording to an adjustably installation of the camera C1 and the cameraC2.

FIGS. 5 and 6 are views illustrating an object sensed by a sensing unitof an electronic apparatus according to an embodiment of the presentinventive concept. One or more icons (or menus) Ma, Mb, . . . , Mn maybe displayed on the display area DA. the virtual area VA is disposedover the display area in a perpendicular direction. When an object, forexample, a user's finger Pob is disposed over the virtual area VA ormoves toward or away from the virtual area VA. The sensing unit 203senses the object disposed in the virtual area VA, and the controller210 may determine whether a location (or position) of the object isdisposed to correspond to one of the icons Ma, Mb, . . . , Mn of thedisplay area DA.

When the controller 210 determines that the object is disposed at alocation (or position) Vpa corresponding to the icon Ma of the displayarea DA, the controller 210 may determine that the icon Ma is selectedso that a corresponding function of the icon Ma can be performed. Thecontroller 210 may determine the presence or existence of the object inthe virtual area VA as a user input. The object can be represented by astationary object disposed at a position or a movable object movablebetween two positions. The controller 210 may control the input/outputunit 204 to display a character or sign Tpa within the correspondingicon Ma of the display area DA to represent that the icon Ma isselected. It is possible that the icon Ma is highlighted or emphasized.The controller 210 processes the function corresponding to the userinput and controls the input/output unit 204 to generate an output, forexample, displaying an image corresponding to the processed function,communicating with other device, or transmitting or receiving dataaccording to the processed function.

Since the display area DA includes a touch screen, the controller 210may receive a user input from a touch input though the input/output unit204 and/or a sensed object though the sensing unit 203. For example, atouch input on the icon MA of the display area DA and a sensed object inthe virtual area are determined as a user input to control theelectronic apparatus.

FIGS. 7 and 8 are views illustrating a point of a virtual area tocorrespond to a display point of a display area in an electronicapparatus according to an embodiment of the present inventive concept.An object, for example, a user finger Pob, moves toward or disposed at aposition VPa or VPb with respect to a display center line DCL of thedisplay area DA in the virtual area VA. The positions VPa and TPa ofFIG. 7 are opposite to the positions VPb and TPb of FIG. 8 with respectto the display center line DCL of the display area DA. All of thepositions VPa, VPb, TPa, TPb are disposed in a side of the sensingcenter line SCL of the sensing unit 203.

FIGS. 9 and 10 are views illustrating a status of an objectcorresponding to a virtual point with respect to a virtual area and adisplay area in an electronic apparatus according to an embodiment ofthe present inventive concept. When an object moves between twopositions with respect to a position VPa in a first direction of FIG. 9or in a second direction of FIG. 10 , the controller 210 determines thepresence, existence, or movement of the object as a user input to selectthe icon Ma, and then perform the corresponding function. Although FIGS.9 and 10 illustrate two directions, the present general inventiveconcept is not limited thereto. The controller 210 may detect themovement of the object in a direction other than the two directions ofFIGS. 9 and 10 .

FIGS. 11 and 12 are views illustrating objects sensed by a sensing unitin an electronic apparatus according to an embodiment of the presentinventive concept. When the sensing unit 203 senses two or more objects,the controller 210 may provide a priority to at least one of objects todetermine a user input. For example, when there are two objects in thevirtual area VA, the controller 210 may determine that a user finger Pobhas a higher priority than a user face as a user input. It is alsopossible that a distal end T of the user finger Pob can have a higherpriority than other portions of the user finger Pob as a user input. Theuser input can be determined according to the presence, existence, andmovement of the distal end T in the virtual area.

FIG. 13 is a view illustrating a location of a sensor of a sensing unitin an electronic apparatus according to an embodiment of the presentinventive concept. The sensing unit may be a camera having a lens unit,and the lens unit may have a photographing angle to define a sensingarea SA and a sensing center line SCL disposed at a center of thesensing area SA. The sensing area SA may have two sensing areas SAa andSAb which are disposed opposite to each other with respect to thesensing center area SCL. The lens unit may have a sensor which may notdisposed at a center area but disposed at a side Sa or Sb with respectto the sensing center line SCL to sense one of the two sensing areas SAaand SAb. In this case, one of the two sensing areas SAa and SAb may notbe sensed by the sensor disposed at the side.

FIGS. 14A, 14B, 15A, and 15B are views illustrating a virtual area of asensing unit and a display area of a display unit according to anembodiment of the present inventive concept. The virtual area VA todetect an object as a user input is disposed at one side of the sensingarea SA, for example, the virtual plane VP, with respect to the sensingcenter line SCL. The sensing area SA and the display area DA areillustrated in a vertical direction in FIGS. 14A and 14B and in ahorizontal direction in FIGS. 15A and 15B, according to a location ofthe sensing unit with respect to the display area DA.

FIG. 16 is a view illustrating a location of a sensing unit in anelectronic apparatus according to an embodiment of the present inventiveconcept. The sensing unit may be disposed at a center of the displayarea DA or at a position on the display center line DCL and/or thesensing center line SCL. The lens unit may be disposed at a bottom DPbof a display panel (touch panel) DP. The display panel DP may be aliquid crystal device (LCD) having a screen corresponding to the displayarea DA, a circuit element to form an image to be displayed on thescreen, and a light element to generate light toward the circuitelement. The sensing unit may be disposed at a center of the LCD. Inthis case, the LCD may provide a transparent condition to a portioncorresponding to the sensing unit such that the sensing unit senses theobject through the LCD. The light element may have a portioncorresponding to the sensing unit to prevent interference with thesensing operation of the sensing unit. The display panel DP may beformed with an LED unit having a number of LEDs. In this case, the LEDunit may have a portion without an LED to prevent interference with theoperation of the sensing unit.

FIG. 17 is a view illustrating a virtual area of a sensing unit and adisplay area of a display unit according to an embodiment of the presentinventive concept. When the sensing unit is disposed at the center ofthe display area DA, the virtual area VA is disposed in the display areaDA, and the sensing center line SCL and the display center area SCL areidentical to each other.

FIGS. 18A and 18B are views illustrating a virtual point displayed on adisplay area of a display unit in an electronic apparatus according toan embodiment of the present inventive concept. When an object Ob issensed to correspond to one of icons Ma, Mb, and Mc, a symbol SYM isdisplayed at the corresponding icon Ma, Mb, or MC on the display areaDA, as illustrated in FIG. 18A. The symbol SYM representing the sensedobject may be disposed at a center portion or a side portion of an areaof the corresponding icon Ma, Mb, and Mc. When the object is sensed tobe disposed at a portion which does not correspond to one of the iconsMa, Mb, and Mc, it is possible that a symbol SYM may be displayed at apoint outside the icons MA, Mb, or Mc, as illustrated in FIG. 18B. Thesymbol SYM may be a dot, a flash, a brink, or a sign. However, thepresent general inventive concept is not limited thereto. The symbol maybe a characteristic change of the corresponding one of the icons Ma, Mb,and MC while the other icons do not show any characteristic change.

FIGS. 19 and 20 are flowcharts illustrating a method of inputting a usercommand using a sensing unit in an electronic apparatus according to anembodiment of the present inventive concept.

Referring to FIG. 19 , when a sensing unit, for example, a camera, isturned on, activated, or starts one of functions thereof in operation1910, an object exists in the photographed image in operation 1920. Avirtual area is extracted from the photographed image in operation 1930.An object is extracted from the virtual area in operation 1940. Thestatus of the object is analyzed and determined as a user input when theobject is disposed at a position corresponding to an icon displayed on adisplay area in operation 1950. A function corresponding to the icon isperformed in operation 1960.

Referring to FIG. 20 , when a sensing unit, for example, a camera, isturned on, activated, or starts one of functions thereof in operation2010, an object is detected from a virtual area corresponding to adisplay area in operation 2020. A coordinate of the virtual areacorresponds to a coordinate of the display area. When a position of theobject with respect to the coordinate of the virtual area corresponds toa position of an icon with respect to the coordinate of the displayarea, it is determined that the presence, existence, or movement of theobject is a user input in operation 2030. A function of the icon isperformed in operation 2040.

FIG. 21 is a view illustrating a location of a sensing unit in anelectronic apparatus 2100 according to an embodiment of the presentinventive concept. The sensing unit may be one or more sensing elementsdisposed positions 2103 a, 2103 b, 2013 c, and 2103 d around a displayarea DA of a display panel 2102 on a housing 2101. When one or morecameras are included in the sensing unit, the cameras may be installedat positions 2104 and at least one of the positions 2103 a, 2103 b, 2013c, and 2103 d. It is possible that the sensing element and the cameramay be same to photograph an object as a photographing mode and to sensean object as a user input with respect to the display area DA in asensing mode. Accordingly, the camera can be usable in one of thephotographing mode and the sensing mode or also usable in both thephotographing mode and the sensing mode at the same time. In this case,an image corresponding to the photographed object has a centercorresponding to the sensing center line SCL, and an image correspondingto the sensed object in the virtual area VA may have a centercorresponding to the display center line SCL. When the object disposedat a position on the display center line DCL is photographed anddisplayed at a position different from the display center line DCL.However, when the object disposed at a position on the display centerline DCL is sensed and analyzed to be disposed at a position on thedisplay center line SCL according to coordinates of the display area DAand the virtual area VA.

FIGS. 22 and 23 are views illustrating an electronic apparatus having asensing unit installed thereon according to an embodiment of the presentgeneral inventive concept. FIG. 22 illustrates that a sensing unit isinstalled at a side of the display area DA such that a sensing centerline SCL is perpendicular to the display area DA and a boundary line BLis inclined to spaced apart from the display area according to adistance from the sensing center line SCL or the sensing unit. FIG. 23illustrates that a sensing unit is installed at a side of the displayarea DA such that a sensing center line SCL is not perpendicular to thedisplay area DA. The sensing unit may be installed such that a boundaryline BL can be substantially parallel to the display area DA. However,the present general inventive concept is not limited thereto. Theboundary line BL may be spaced apart from the display area DA by adistance so that the sending unit can sense a user finger or tooldisposed adjacent to the display area DA. The sensing unit may have asub-housing with a lens unit having the sensing center line SC, and thesub-housing may be installed in the sub-housing at an inclination anglewith a plane of the display area DA. A method of installing a sensingunit on a housing is well known, detail description thereof is omitted.It is possible that the boundary line BL of FIG. 23 may be inclined withrespect to the display area DA. In this case, an inclination angle IAbof the boundary line BL of FIG. 23 is smaller than an inclination angleIAa of the boundary line BL of FIG. 22 . A hatched portion is a virtualarea of a sensing area disposed above the display area DA in a directionperpendicular to the display area DA.

The present general inventive concept can also be embodied ascomputer-readable codes on a computer-readable medium. Thecomputer-readable medium can include a computer-readable recordingmedium and a computer-readable transmission medium. Thecomputer-readable recording medium is any data storage device that canstore data as a program which can be thereafter read by a computersystem. Examples of the computer-readable recording medium include asemiconductor device, a read-only memory (ROM), a random-access memory(RAM), a flash memory, a semiconductor chip package, CD-ROMs, magnetictapes, floppy disks, and optical data storage devices. Thecomputer-readable recording medium can also be distributed over networkcoupled computer systems so that the computer-readable code is storedand executed in a distributed fashion. The computer-readabletransmission medium can transmit carrier waves or signals (e.g., wiredor wireless data transmission through the Internet). Also, functionalprograms, codes, and code segments to accomplish the present generalinventive concept can be easily construed by programmers skilled in theart to which the present general inventive concept pertains.

As described above and also illustrated in the drawings, a user mayinput a user command on a virtual area corresponding to a display areawithout touching the display area in an electronic apparatus accordingto an embodiment of the present inventive concept. Also, the virtualarea may be spaced apart from the display area, disposed parallel to thedisplay area, and disposed to overlap the display area in a verticaldirection from the display area in the electronic apparatus such that anobject can be sensed to correspond to the same point in the virtual areaand the display area in the vertical direction according to anembodiment of the present general inventive concept.

Moreover, as described above and also illustrated in the drawings, theelectronic apparatus may include a housing, a display unit installed inthe housing and having a display area to display an image or a menuicon, a sensing unit installed in the housing adjacent to the displayunit, and having a virtual area to sense an object which is disposedover the display area of the display unit, the virtual area defined by aboundary extended from the display area in a direction perpendicular tothe display area, and a controller to analyze status of the objectaccording to an original state of the object and a sensed state of theobject, and to determine the analyzed status of the object as a userinput.

The original state of the object and the sensed state of the object withrespect to the display area may be different from each other accordingto a difference between center lines of the sensing unit and the displayarea of the display unit.

The controller may adjust the sensed state of the object to the originalstate of the object with respect to the display area to determinewhether the status of the object corresponds to the user input to thedisplay area.

The controller may control the display unit to display a symbolrepresenting the status of the object with respect to the display area.

The symbol may be displayed on the menu icon or outside the menu icon inthe display area according to a location of the status of the objectwith respect to the display area.

The virtual area and the display area may overlap in the boundary in theperpendicular direction.

The menu icon may be disposed in the display area, and the object may bedisposed in the virtual area to correspond to the menu icon of thedisplay area.

The sensing unit may have a sensing area with a sensing center line andthe display area has a display center line which is spaced apart fromthe sensing center line by a distance.

The virtual area may be disposed in a side of the sensing area withrespect to the sensing center line, and the controller may extract thevirtual area from the sensing area.

When the object moves a first distance in a first direction within thevirtual area, the sensing unit may sense the object moving a seconddistance in a second distance since a sensing center line of the sensingunit is not a center line of the virtual area, and the controller mayanalyze the status of the object according to differences between thefirst and second distances and between the first and second distancesand determines the status of the object as the user input with respectto the display area.

When the object moves toward a first position of the display area, thesensing unit may sense the object moving toward a second position of thedisplay area in the virtual area, the second position being differentfrom the first position. The controller may adjust the second positionto the first position with respect to the display area.

The sensing unit may sense the object at a first position with respectto the virtual area, and the controller may convert the sensed firstposition into a second position with respect to the display area as thestatus of the object and determines the status of the object as the userinput.

The display area and the virtual area may have a same coordinate, thesensing unit may sense the object in the coordinate of the virtual area,and the controller may determine the status of the object as the userinput from the sensed object according to the coordinate of the displayarea.

The virtual area is disposed to overlap the display area in a verticaldirection from the display area, and the controller may determinewhether the status of the sensed object of the virtual area correspondsto the menu icon of the display area, and also determine the status ofthe sensed object as the user input of the menu icon to perform afunction of the menu icon.

The sensing unit may sense one or more sub-objects, and the controllermay determine one of the one or more sub-objects as the object.

The sensing unit may sense a user face and a user finger, and thecontroller may determine the user finger as the object.

The sensing unit may sense a user finger as the object, and thecontroller determines a distal end of the user finger as the status ofthe object.

The sensing unit may have a sensing center line to have an angle with adisplay center line of the display area, and the sensing center line ofthe camera may not be parallel to the display center line of the displayarea.

The sensing unit may be a camera having a photographing area tophotograph the object. The controller may extract from the photographingarea the virtual area including the object disposed over the displayarea, and may also determine from the photographed image that the objectis disposed over the menu icon in a direction perpendicular directionfrom the display area.

The display unit may be a touch panel usable as an output unit todisplay the image or the menu icon and an input unit to receive the userinput to control the image or the menu icon. The sensing unit mayinclude a lens unit with a sensing area in which the virtual area and anon-virtual area are included, and the lens unit includes a sensor tophotograph the virtual area.

The display unit may include a display panel having the display area,and the sensing unit is disposed at a bottom area of the display panelof the display unit.

Furthermore, as described above and also illustrated in the drawings,The electronic apparatus may include a housing, a display unit installedin the housing, and having a display area usable as an output unit todisplay an image or a menu icon and an input unit to receive a userinput to control a function of the electronic apparatus including theimage or the menu icon, a sensing unit installed in the housing adjacentto the display unit, and having a camera with a photographing areaincluding a virtual area to sense an object which is disposed over thedisplay area of the display unit, the virtual area defined by a boundaryextended from the display area in a direction perpendicular to thedisplay area, and a controller to analyze status of the object accordingto an original state of the object and a sensed state of the object, andto determine the analyzed status of the object as the user input.

Furthermore, as described above and also illustrated in the drawings,the electronic apparatus may perform a method including providing ahousing with a display unit having a display area to display an image ora menu icon, and a sensing unit installed adjacent to the display unitand having a virtual area to sense an object which is disposed over thedisplay area of the display unit, the virtual area defined by a boundaryextended from the display area in a direction perpendicular to thedisplay area, and analyzing status of the object according to anoriginal state of the object and a sensed state of the object, anddetermining the analyzed status of the object as a user input.

Furthermore, as described above and also illustrated in the drawings,the present general inventive concept may be implemented as acomputer-readable medium to contain computer-readable codes as a programto perform the method described above in the electronic apparatus.

Although a few exemplary embodiments of the present general inventiveconcept have been shown and described, it will be appreciated by thoseskilled in the art that changes may be made in these exemplaryembodiments without departing from the principles and spirit of thegeneral inventive concept, the scope of which is defined in the appendedclaims and their equivalents.

What is claimed is:
 1. A mobile device, comprising: a housing; a networkinterface to communicate with an external network; a user interfacecomprising a panel with a display panel to display a menu image and atouch panel to receive a touch input, the panel comprising a firstportion and a second portion in the first portion, the second portionhaving an interference-prevention condition; a sensing unit comprisingfirst and second front cameras disposed in the second portion of thepanel to sense an object over the panel through the second portion; anda control unit to control the sensing unit in a photographing mode and asensing mode to perform a function of the mobile device.
 2. The mobiledevice of claim 1, wherein the object is distanced from the panel. 3.The mobile device of claim 1, wherein the first portion and the secondportion are parallel to each other within the panel.
 4. The mobiledevice of claim 1, wherein the second portion is smaller than the firstportion with respect to a display area of the panel.
 5. The mobiledevice of claim 1, wherein the second portion is surrounded by the firstportion.
 6. The mobile device of claim 1, wherein the second portioncomprises opposite sides with the first portion.
 7. The mobile device ofclaim 1, wherein the first portion includes light elements, and thesecond portion does not include the light elements.
 8. The mobile deviceof claim 1, wherein the first portion comprises a liquid crystal device.9. The mobile device of claim 1, wherein the first portion comprises alight emitting device.
 10. The mobile device of claim 1, wherein thefirst portion comprises a first number of light elements, and the secondportion comprises a second number of light elements.
 11. The mobiledevice of claim 1, wherein the interference-prevention conditioncomprises a transparent condition.
 12. The mobile device of claim 1,wherein the interference-prevention condition prevents the first andsecond front cameras from being interfered during sensing the object inthe photographing mode and the sensing mode.
 13. The mobile device ofclaim 1, wherein the first front camera photographs the object throughthe second portion in the photographing mode.
 14. The mobile device ofclaim 13, wherein the control unit controls the display panel to displaythe photographed object.
 15. The mobile device of claim 1, wherein thesecond front camera senses the object the second portion in the sensingmode.
 16. The mobile device of claim 15, wherein the control unitcontrols the display panel to display a symbol representing a status ofthe object.
 17. The mobile device of claim 1, wherein the first frontcamera and the second front camera sense the object as a user input inthe sensing mode.
 18. The mobile device of claim 1, wherein the sensingunit is activated to photograph and extract an object image from thephotographed image as a user input to perform a second function of themobile device in the sensing mode.
 19. The mobile device of claim 1,wherein the sensing unit selectively performs the sensing mode and thephotographing mode.
 20. The mobile device of claim 1, wherein thecontrol unit controls the first and second front cameras to selectivelyperform the sensing mode and the photographing mode.
 21. The mobiledevice of claim 1, wherein the object comprises a plurality ofsub-objects, and the control unit performs a second function of themobile device according to one sensed sub-object with a priority. 22.The mobile device of claim 1, wherein the object comprises a user havinga plurality of portions, one of which has a higher priority, and thecontrol unit performs a second function of the mobile device accordingto one sensed sub-object with the higher priority.
 23. The mobile deviceof claim 1, wherein the object comprises a user having a plurality ofportions, one of which has a priority, and the control unit performs asecond function of the mobile device according to the sensed user andthe sensed one portion with the priority.
 24. The mobile device of claim1, wherein the first front sensing camera comprises a first lens, thesecond front comprises a second lens, and the first lens and the secondlens are disposed in the second portion.
 25. The mobile device of claim24, wherein the first lens and the second lens have first and secondsensing center lines, respectively, which are parallel to a lineperpendicular to a display surface of the panel.
 26. The mobile deviceof claim 24, wherein the first lens has a first center line which isparallel to a line perpendicular to a display surface of the panel. andthe second lens has a second center line which is not parallel to theline perpendicular to the display surface of the panel.
 27. The mobiledevice of claim 24, wherein at least one of the first lens and thesecond lens has a sensing center line which is not parallel to a lineperpendicular to a display surface of the panel.